Universal Mosaicing using Pipe

Video mosaicing is commonly used to increase the visual eld of view by pasting together many video frames. Existing mosaicing methods are eeective only in very limited cases where the image motion is almost a uniform translation or the camera performs a pure pan. Forward camera motion or camera zoom are very problematic for traditional mosaicing. A mosaicing methodology to allow image mosaicing in the most general cases is presented, where frames in the video sequence are transformed such that the optical ow becomes parallel. This transformation is an oblique projection of the image into a viewing pipe whose central axis is the trajectory of the camera. The pipe projection enables to deene high-quality mosaicing even for the most challenging cases of forward motion and of zoom. In addition, view interpolation, generating dense intermediate views, is used to overcome parallax eeects. This work describes, for the rst time, a methodology that enables mo-saicing of video frames for almost any camera motion, including forward motion and zoom. The pipe projection presented here preserves all details in the sequence with their best resolution. Importance As a single frame represents only a fraction of a scene, combining frames into a panoramic mosaic gives a more complete representation of the scene. Unfortunately current mosaicing methods were applicable only for very limited camera motions. These limitations are removed using the pipe mosaics. The closest work is mosaicing by projection into a cylinder (McMillan & Bishop, Mann & Picard), which is limited to camera pan only. Also the mosaicing work at Sarnoo (Irani, Anandan, & Hsu, etc.) which is mainly limited to scenarios like nadir aerial images or a very small camera rotation. Rather than aligning images to each other, the pipe projection transforms the images based on the camera motion such that the optical ow becomes parallel. After this, alignment and mosaicing becomes very simple. Use of the results of this work Describing a complete scene from a video sequence can be used in visu-alization, video indexing, and video compression.